2 links tagged with all of: alignment + constitutional ai + anthropic
Click any tag below to further narrow down your results
Links
Leading AI companies are hiring philosophers to craft constitutional rules that guide AI behaviour. These experts debate between deontological and other ethical frameworks to ensure consistent, principled actions from systems deployed in homes and public spaces.
- Anthropic, OpenAI, and Google DeepMind have each hired dozens of philosophers to write "constitutions" governing AI behavior, favoring different ethical frameworks (Anthropic's Kantian deontology vs. DeepMind's utilitarian harm-scoring system).
- These philosophers work directly with engineers to convert abstract principles into training data and reward signals, not just theoretical writing.
- OpenAI tracks rule violations per thousand queries, targeting under 0.5 for high-risk categories like medical or legal advice.
- Critics argue these philosophers lack technical expertise and that the resulting AI constitutions still rely on opaque enforcement mechanisms.
The article discusses the new version of Claude's constitution, which outlines explicit values for AI behavior. It explains how Constitutional AI improves upon traditional human feedback by using AI-generated principles to ensure safer and more transparent model outputs. The principles aim to address ethical concerns while allowing for continuous improvement.
- Constitutional AI replaces most human feedback with AI self-critique against a written constitution, then AI-generated reinforcement learning for harmlessness, cutting the need for people to review toxic content.
- Claude trained this way got better at handling adversarial prompts without becoming less helpful, and the approach makes the values steering the model's outputs more transparent.
- The constitution draws on sources like the UN Declaration of Human Rights and other labs' safety practices, but Anthropic admits it's still skewed toward Western viewpoints and remains a work in progress shaped by trial and error.